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© Improved method and apparatus for reconstructing missing color. 

© An improved method and apparatus is provided 
for sensing scene light and providing sampled image 
data in three colors in response thereto. The sam- 
pled image data is subsequently interpolated for the 
non-sampled colors and thereafter subtracted to pro- 
vide two color difference signals. The two color 
difference signals, in turn, are each median filtered 
and subsequently reconstructed in conjunction with 
the originally sampled image data to provide an 
image of the subject with substantially reduced color 
fringing. The median filter preferably filters color 
difference signals corresponding to a select number 
of image sensing elements separated with respect to 
each other by non-select image sensing elements. 
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IMPROVED METHOD AND APPARATUS FOR RECONSTRUCTING MISSING COLOR SAMPLES 



Field of the Invention 

This invention relates generally to an improved 
apparatus and method for sensing and interpolating 
image data and, more particularly, to an improved 
apparatus and method for sensing sampled image 
data and thereafter interpolating for the nonsam- 
pled image data in a manner that substantially 
reduces color fringing. It will be convenient to de- 
scribe the invention more particularly with respect 
to apparatus that detects and reproduces an image 
by means of three colors, the most widely used 
format for color imaging. 



Background of the Invention 

Electronic imaging cameras for recording still 
images are well known in the art. Such cameras 
can record a plurality of still images on a single 
magnetic disk or tape in either analog or digital 
format for subsequent playback on any well-known 
cathode ray tube viewing device. Printers may also 
be utilized with such cameras in a well-known 
manner to provide hard copy of the recorded im- 
ages. Such electronic imaging still cameras may 
utilize two-dimensional image sensing arrays such 
as charge coupled devices (CCD's) which integrate 
incident scene light over a predetermined time to 
provide an electronic information signal corre- 
sponding to the scene light intensity incident on 
the array. Such two-dimen sional image sensing 
arrays comprise a predetermined number of dis- 
crete image sensing elements or pixels arranged in 
a two-dimensional array in which each image sens- 
ing element responds to incident illumination to 
provide an electronic information signal corre- 
sponding to the intensity of the incident illumina- 
tion. 

In order to record color images, the illumination 
incident on the two-dimensional image sensing ar- 
ray is filtered so that different image sensing ele- 
ments receive different colored illumination. The 
filters are arranged in well-known patterns across 
the face of the image sensing array, such as a 
repeating pattern of red, green and blue stripes. 
Alternatively, individual image sensing elements or 
pixels across each line may be filtered in a repeat- 
ing pattern of red, green, blue, green filters, as is 
well known in the art. Since each image sensing 
element can only detect one color of illumination, 
the color information for the other colors not de- 
tected by that image sensing element must be 
filled in. Riling in the missing color information is 



generally accomplished by interpolating the de- 
tected image data for each color to determine color 
values for all the colors for each image sensing 
element In particular, there results three groups of 

5 image sensing elements of which respective ones 
have measured values for one given color and an 
interpolated value for the other two colors. 

Conventional types of interpolation, however, 
can provide images with objectionable aliasing arti- 

70 facts such as "color fringes" near sharp edges. 
The conventional approach to solve this problem is 
to eliminate the color fringes at the expense of 
image sharpness by either blurring the picture or 
removing selected spacial frequencies from the 

75 picture (antialiasing) so that edges do not create 
color fringes. Blurring the image in this manner, 
however, has its obvious disadvantages resulting in 
a reduction in resolution and a so-called "fuzzy" 
picture. 

20 In my earlier U.S. Patent No. 4,663.655, which 

issued on May 5, 1987, there is described a novel 
technique for providing the desired interpolated val- 
ues. In this technique, there is first inserted for the 
missing values approximate values derived by sim- 

25 pie linear interpolation in the usual fashion, and 
thereafter, the resulting values are used to derive a 
number of difference values for each of the pixels 
by subtracting the resulting values at different pairs 
of colors. These difference values are then passed 

30 through a simple linear median filter to reduce the 
color fringe artifacts described above, and then the 
various filtered differences are combined to re- 
construct three color values at each pixel for use in 
the image scene reproduction. 

35 While this technique provides a considerable 

improvement, each color image signal processed 
requires a separate median filter of a relatively 
complex nature. 

Accordingly, it is desirable to eliminate color 

40 fringing artifacts in the aforementioned manner us- 
ing fewer median filters of a less complex nature. 

SUMMARY OF THE INVENTION 

45 

The present invention represents an improve- 
ment of the apparatus of my earlier invention pri- 
marily in the use of fewer median filters of a less 
complex nature, 
so First, in the reproduction of a three-color image 

scene, there are used only two rather than three 
difference signals that require median filtering for 
removing the color fringe artifacts. The image may 
be satisfactorily reconstructed, despite the elimina- 
tion of one of the median filters previously used. 
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This aspect of the invention is independent of the 
median filter used and thus may be embodied in 
apparatus using any median filter that replaces a 
sample with the median of neighboring samples. 

Second, the efficacy of the median filter used 
in the apparatus of this invention is, substantially, 
as dependent on the spatial extent of the pixels 
used as it is on the number of pixels used, so that 
a linear median filter which is seven pixels wide but 
uses only three terms, the two end pixels and the 
center pixel whose value is to be replaced with the 
median value of the three pixels when combined 
with low pass filtering, is substantially as effective 
as an eleven-pixel wide eleven-term median filter. 

Moreover, for two-dimensional filtering in ac- 
cordance with my invention, there is employed an 
area median or two dimensional filter which uses 
only nine pixels chosen from an area seven pixels 
wide and five pixels high. This filter when com- 
bined with two-dimensional low pass filtering com- 
pares favorably with one that uses the entire array 
of thirty five pixels of the area. These improve- 
ments are similarly independent of the number of 
median filters used. 

In particular, an illustrative embodiment of the 
present invention includes a conventional solid- 
state image sensing camera, including a CCD, in 
which a different one-third of the total number of 
pixels, or discrete sensing elements, views the 
image scene in respective ones of three different 
wavelength ranges, or colors, and are provided with 
a measured signal value at each such pixel. There 
is next provided for each pixel an interpolated 
signal vaiue for each of its two unmeasured colors, 
equal to that of the measured range of 
wavelengths. Thereafter, there are derived for each 
pixel a first difference signal value that is a mea- 
sure of the difference in intensity of its signal for 
one pair of colors and a second difference signal 
that is a measure of the difference in intensity of its 
signal value for a different pair of colors. There- 
after, each of these two difference signal values is 
passed through a novel form of median filter. For 
one-dimensional filtering, the filter is preferably 
seven elements wide but uses only the first, fourth 
and seventh elements to find the value to be sub- 
stituted for the value of the fourth element. For two- 
dimensional filtering, the filter is preferably seven 
elements wide and five elements high but uses 
only nine of the elements to find the value to be 
substituted for the center element. 

Finally, from these two filtered difference signal 
values and the original measured signal values, 
there are reconstructed for each pixel three signal 
values, one for each of the three different colors. 
The reconstructed signal values are then used to 
reproduce the original image scene. 



Detailed Description of the Drawings 

The invention will be better understood from 
the following more detailed description taken in 
5 conjunction with the accompanying drawings. 

FIG. 1 is a pictorial flow chart illustrating 
steps of the basic interpolation process of the 
invention; 

FIG. 2 is a schematic block diagram of an 
10 illustrative embodiment of the invention; 

FIG. 3 shows in schematic block form a one- 
dimensional median filter for use in the embodi- 
ment of FIG. 2; and 

FIG. 4 shows the masking pattern of a two- 
15 dimensional filter for use in the embodiment of FIG. 
1. 



Detailed Description of the Invention 

20 

Referring now to FIG. 1, there is shown in 
pictorial form a flow diagram of the process of 
interpolation in accordance with the present inven- 
tion. 

25 In frame FrA, there is depicted a coiored scene 

which can be viewed as made up of the super- 
position of three images, each limited to one of 
three ranges of wavelengths, for example, yellow, 
green and cyan. Typically, the colored scene light 

30 is passed through a repeating array of three sepa- 
rate vertical filters. The filtered scene light is then 
scanned by an electronic imaging sensor, for ex- 
ample, a charge-coupled device (CCD) of the 
frame or line transfer type. As will be readily under- 

35 stood, the colored filters may be integrally formed 
on the CCD. in the usual fashion, each pixel of the 
imaging sensor receives only one of the three 
colors, and in each triplet of three horizontally 
contiguous pixels, each pixel receives a different 

40 color. This is depicted in frame FrB, which shows 
the scene viewed by successive pixels of a CCD 
sensor, each in only one of the three colors. In 
frame FrC, there is shown the image scene as 
divided into, three separate scenes, one in each 

45 channel of a color C, G and Y, with intensity values 
provided at only every third pixel for each scene, 
or one pixel of each triplet. An intensity value at the 
other two pixels of a triplet for such color is ob- 
tained by a first interpolation. Since this first inter- 
so polation is not critical, a simple interpolation is 
possible. For example, it is adequate merely to use 
the vaiue measured for a given color at one pixel of 
a triplet for a value of that color for the two other 
pixels of the triplet. As a result, the three yellow 

55 values for the three pixels of a triplet are the same, 
the three green values are the same, and the three 
cyan values are the same. This is represented in 
Frame FrD. 
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The signals in each color channel are then 
amplified, the relative gains of the three channels 
being such that a neutral grey light input elicits the 
same response from all three channels. 

There are then performed color signal subtrac- 
tions by a subtractor 11 to derive color signal 
difference values. Y-G and C-G. The amplitude of 
the signal in the G channel is subtracted from that 
of the Y channel and the C channel, respectively. 
The result of this subtraction is represented sche- 
matically in the top two channels of frame FrE. The 
third and bottom channel remains the same as 
frame FrB. 

In the foregoing respects, the process of the 
present invention resembles that described in my 
aforementioned patent for three color imaging, ex- 
cept it will be readily understood that only two 
color subtractions are made for a three color sys- 
tem in comparison with the three color subtractions 
required by my aforementioned patent. 

In the apparatus of my prior patent, the result- 
ing three outputs of the subtractors were then each 
subjected to a one-dimensional horizontal linear 
median filter which was five elements or samples 
wide and which used all five samples. As is dis- 
cussed in the patent, this step is important to 
reduce color fringing artifacts that otherwise are 
present. 

However, in practice it was found that while this 
horizontal linear filter using five samples made for 
a discernible reduction in artifacts, it also required 
fairly complex circuitry to implement. It has been 
found in accordance with the method and appara- 
tus of this invention that at least as good a result 
can be obtained with considerable simplification in 
circuitry if there is used for one-dimensional 
median filtering simply a three term median filter in 
which the three terms comprise the sample in- 
volved plus the two samples three pixels away 
from it on either side in the same horizontal line. 
Moreover, still further improvement can be ob- 
tained if there is employed a two-dimensional area 
median filter which uses the median of nine sam- 
ples spaced apart over an area encompassing thir- 
ty five possible samples. 

The two filtered-difference sets of signal values 
are combined at 13 with the original set of signal 
values as depicted by Frame FrF to reconstitute 
three color signal values for each pixel. These 
values are then combined at 15 to produce the 
original image with the three additive primary col- 
ors, red. green and blue in the usual fashion, as 
depicted by frame FrG. 

Referring now to FIG. 2, there is shown in 
block schematic form apparatus 100 in accordance 
with the present invention. First, there is derived 
from an image sensor 10. typically a CCD. a signal 
in serial form in which successive samples repre- 



sent the values of charge collected at successive 
pixels of the sensor, where each pixel is excited by 
only one of the three color components since the 
illumination on each pixel first passes through a 

5 vertical line filter (not shown) that transmits only 
one of such colors as previously discussed. Prefer- 
ably, there is an array of such filters in repeating 
sets of three, transmitting cyan, green and yellow 
components, respectively. These are preferably ar- 

70 ranged so that sets or triplets of three successive 
pixels will experience respective ones of the three 
filters, as is conventional in color imaging systems. 
This measured signal value on each pixel is then 
supplied to respective ones of three separate sam- 

75 pie and hold circuits 12, 14 and 16, each of which 
is designed to store the measured value at a dif- 
ferent one of each pixel of a triplet. These cor- 
respond, respectively, to channels for the yellow- 
excited pixels, the green-excited pixels, and the 

20 cyan-excited pixels. Each sample and hold circuit 
holds such sample for the scanning time interval of 
the next two succeeding pixels to permit process- 
ing with such sample until it receives a new sample 
from a pixel excited in its characteristic color. Cir- 

25 cuits for accomplishing this sample and hold func- 
tion are well known in the art. 

By holding each sample for the next two suc- 
ceeding pixel intervals, there is essentially repli- 
cated the signal value for the one pixel which 

30 provided the original measured signal value as well 
as the two succeeding pixels which were initially 
free of signal values for that color. This serves the 
function of the linear interpolation described in the 
aforementiond patent. Alternatively, linear interpola- 

35 tion as described in my prior patent may be substi- 
tuted for such replication interpolation. 

The outputs of each of these sample and hold 
circuits 12, 14 and 16 are passed, respectively, to 
amplifiers 18, 20 and 22 where each is amplified. 

40 The gain of the different amplifiers is adjusted, as 
previously discussed, so that a neutral color, e.g., 
white, after conversion by the camera to three 
signal samples of different colors will produce sig- 
nals of equal amplitude at each of the three am- 

45 plifier outputs. Typically, this may involve a greater 
gain for the amplifier of the green channel than the 
others because of the greater losses in green to 
filtering. 

Thereafter, the outputs of amplifiers 18 and 20 
so are supplied to a subtractor network 24 for deriving 
the difference signal value Y-G, and the outputs of 
amplifiers 20 and 22 are supplied to subtractor 
network 26 for deriving the difference signal value 
C-G, where C, G and Y are the signal values in the 
55 C, G and Y channels, respectively. The subtractive 
networks may simply be differential amplifiers with 
inverting and non-inverting inputs. 

The difference signals available at the outputs 
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of subtractors 24 and 26 are preferably passed 
through low pass filters 28 and 30. respectively, for 
some blurring of the signals. 

The two separate difference signals are then 
supplied to the median filters 32 and 34, respec- 
tively, which in accordance with the present em- 
bodiment are preferably one-dimensional median 
filters seven samples wide that use only three 
samples. In such a filter, there is examined the 
signal values for the first, fourth and seventh pixels 
of a line of seven pixels and the median of the 
three values is chosen as the value for the fourth 
pixel of the line. 

Referring now to FIG. 3, there is shown a 
circuit 200 in accordance with the present invention 
which acts as a one-dimensional median filter. It 
comprises a delay line 51 , one end 52a of which is 
supplied with the signal to be filtered, and the other 
end 52b of which is terminated in its characteristic 
impedance Z c to be reflectionless. Numerals 1 - 7 
are included to indicate schematically seven sig- 
nals along delay line 51. Taps 53. 54 and 55 
indicate the tapping of the signals from the first, 
fourth and seventh pixels. The signals available at 
each of these taps at a given time are supplied to 
the three inputs 57, 58 and 59, respectively, of an 
analog sorting network 60, designed to provide at 
its output at a predetermined time, the value of the 
signal that is the median of the three values of 
signals applied to its three inputs. Various circuits 
may be readily designed for this function. An ex- 
ample of such a circuit is described at pages 72 - 
74 of Electronic Design 2 ( January 18, 1973 in an 
article entitled, "Analog Sorting Network Ranks In- 
puts by Amplitude and Allows Selection." 

Referring again to FIG. 2, after such median 
filtering, the two filtered signals are preferably sup- 
plied, respectively, to low pass filters 36 and 38. 
The outputs of the low pass filters 36 and 38 are 
designated as processed signals. 

These processed signals, Y-G and C-G, in turn, 
are supplied to a reconstructing network 40 which 
uses the two processed signals and the original 
signal provided by the CCD 10, appropriately de- 
layed by a delay element 46, to derive for each 
pixel a signal value at each of the three colors. The 
network 40 comprises a plurality of discrete adders 
and subtractors to which appropriate combinations 
of the two processed difference signals and the 
original signals are applied to derive the desired 
color signal value. In particular, there is derived 
from the reconstructing network 40 three separate 
color outputs, as shown at output terminals Y, G 
and C. 

The network 40 comprises three gate circuits 
shown generally at 50, 52 and 54. Each of the gate 
circuits 50, 52 and 54 is selectively enabled at 
different times by a timing signal control 48 to gate 



three respective yellow, green and cyan image 
information signals between the correspondingly 
designated terminals. The timing control signal 
from circuit 48 controls the gate circuits 50, 52 and 

5 54 so as to enable: gate circuit 50 when color 
information signals are output from the delay 46 for 
pixels filtered to receive green image scene light; 
gate circuit 52 when color information signals are 
output from the delay 46 for pixels filtered to re- 

io ceive cyan image scene light; and, gate circuit 54 
when color information signals are output from the 
delay 46 for pixels filtered to receive yellow image 
scene light 

Thus, each green pixel image value from the 
75 CCD 10 is appropriately delayed by the delay 46 
and sub-sequently transmitted by the gate 50 
which is enabled in the aforementioned manner by 
the time signal control 48 to provide a green image 
information signal at output terminal Gout- 

At the 

20 same time, the processed Y-G signal is added by 
an adder 56 to the green (G) pixel image signal 
received from the CCD 1 0 by way of the delay 46 
to provide a yellow (Y G ) signal value that is subse- 
quently transmitted through the enabled gate circuit 

25 50 to provide a yellow image information signal at 
the output terminal Y out simultaneously with the 
aforementioned green information signal provided 
at output terminal G out . Also at the same time, the 
processed C-G signal is added by an adder to the 

30 same green (G) pixel image signal received from 
the CCD 10 by way of the delay 46 to provide a 
cyan (C G ) signal value that is subsequently trans- 
mitted through the enabled gate circuit 50 to pro- 
vide a cyan image information signal at the output 

35 terminal C ou t simultaneously with the aforemen- 
tioned yellow and green information signals pro- 
vided at output terminals Y out and G out » respec- 
tively. Thus, in this manner after the- delay 46 
transmits green color sample values, the recon- 

40 structing network 40 provides green image informa- 
tion signals corresponding to the green color sam- 
ple values sensed by the CCD 10 and yellow and 
cyan image information signals interpolated in the 
aforementioned manner of this invention for the 

45 same pixels. 

In like manner each cyan pixel image value 
from the CCD 10 is appropriately delayed by the 
delay 46 and subsequently transmitted by the gate 
52 which is enabled in the aforementioned manner 
so by the time signal control 48 to provide a cyan 
image information signal at output terminal C QUt . At 
the same time, the processed C-G signal is sub- 
tracted by a subtractor 60 from the processed Y-G 
signal to provide a Y-C signal. An adder 62 subse- 
55 quently adds the cyan (C) pixel image signal re- 
ceived from the CCD 10 by way of the delay 46 to 
the processed Y-C signal to provide a yellow (Y c ) 
signal value that is subsequently transmitted 
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through the enabled gate circuit 52 to provide a 
yellow image information signal at the output termi- 
nal Y out simultaneously with the aforementioned 
cyan information signal at output terminal C ou t- Ai $o 
at the same time, the same cyan (C) pixel image 
signal received the CCD 10 by way of the delay 46 
is subtracted by a subtracter 64 from the pro- 
cessed C-G signal to provide a green (G c ) signal 
value that is subsequently transmitted through the 
enabled gate circuit 52 to provide a green image 
information signal at the output terminal G 0U t si- 
multaneously with the aforementioned cyan and 
yellow information signals provided at output termi- 
nals Cout and Y outt respectively. Thus, in this man- 
ner after the delay 46 transmits cyan color sample 
values, the reconstructing network 40 provides 
cyan image information signals corresponding to 
the cyan color sample values sensed by the CCD 
10 and yellow and green image information signals 
interpolated in the aforementioned manner of this 
invention for the same pixels. 

Each yellow pixel image value from the CCD 
10 is also appropriately delayed by the delay line 
46 and subsequently transmitted by the gate 54 
which is enabled in the aforementioned manner by 
the time signal control 48 to provide a yellow 
image information signal at output terminal Y ou t. At 
the same time, a subtracter 66 operates to subtract 
the yellow (Y) pixel image signal received from the 
CCD 10 by way of the delay 46 from the pro- 
cessed Y-G signal to provide a green (G Y ) signal 
value that is subsequently transmitted through the 
enabled gate 54 to provide a green image informa- 
tion signal at the output terminal G ou t simultaneous- 
ly with the aforementioned yellow information sig- 
nal at output terminal Y ou t- Also at the same time, a 
subtracter 68 operates to subtract the same yellow 
(Y) pixel image signal received from the CCD 10 
by way of the delay 46 from the signal Y-C output 
from the subtracter 60 to provide a cyan (C Y ) signal 
value that is subsequently transmitted through the 
enabled gate circuit 54 to provide a cyan image 
information signal at the output terminal C out si- 
multaneously with the aforementioned yellow and 
green information signals provided at output termi- 
nals Y out and Gout, respectively. Thus, in this man- 
ner after the delay 46 transmits yellow color sam- 
ple values, the reconstructing network provides yel- 
low image information signals corresponding to the 
yellow color sample values sensed by the CCD 10 
and green and cyan image information signals in- 
terpolated in the aforementioned manner of this 
invention for the same pixels. 

In this manner for each pixel of the CCD 10 in 
which one of three different color values is sensed, 
there are provided interpolated values for the other 
two colors using only two median filters represent- 
ing a substantial simplification in comparison with 
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the median filter interpolation arrangement of my 
aforementioned patent in which three median filters 
were required for a three color system. 

It should be appreciated that modifications in 

5 the processing algorithm used can be made with- 
out departing from the spirit of the invention. In 
particular, the complement of either or both dif- 
ference signals described can be used instead, as 
well as other colored filters such as primary red, 

ro green and blue colored filters. 

tn some instances, the one-dimensional median 
filter will not completely remove the artifacts. If this 
is of concern, a two-dimensional median filter can 
be substituted. In this instance, too, the form of the 

75 filter can be considerably simplified by following 
the principles set forth for the one-dimensional filter 
described. Again, the spatial extent is significant, 
and it is advantageous to provide an increase in 
the spatial extent, while omitting closer pixels to 

20 simplify the number of terms. 

Referring now to FIG. 4, there is depicted a 
seven by five rectangular array 70 of thirty-five 
pixels of which the nine values which are to be 
used in the two-dimensional median filter of the 

25 invention are denoted by P's. The pixel 71 whose 
value is to be replaced with the median value of 
the nine pixels is shown at the center of the array. 
As seen, the eight other pixels include the two on 
the same horizontal line three pixels away on either 

30 side, two on the same vertical line, two pixels away 
on either side, and four pixels on adjacent lines, 
each two pixels away on the horizontal and one 
pixel away on the vertical. It will be well understood 
that the invention is not limited to the specific 

35 spacing arrangement described for the pixels used 
in the median filter operation and that other specific 
arrangements are well within the scope of the in- 
vention. 

in this situation, the sorting of the median value 
40 is done in a well-known manner typically either by 
a special purpose microprocessor or a digital signal 
processor appropriately programmed. 

While the invention has been described for 
apparatus which uses three ranges of wavelength, 
45 or primary colors, for the imaging and reproduction, 
the principles should be applicable to apparatus 
which uses more than three colors. 

Moreover, while the invention has been de- 
scribed primarily with reference to processing ana- 
50 log signals, it should also be amenable to use of 
digital signals if the analog signals derived are first 
converted to digital for processing and reconverted 
to analog for reproduction of the image scene. 

Other embodiments of the invention including 
55 additions, subtractions, deletions and other modi- 
fications of the preferred disclosed embodiment of 
the invention will be obvious to those skilled in the 
art and are within the scope of the following claims, 

6 
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Claims 

1. An electronic imaging apparatus comprising: 
an image sensing array comprising a predeter- 
mined number of discrete image sensing elements 
each responsive to incident illumination from a 
subject to provide an electronic information signal 
corresponding to the intensity of the illumination 
incident thereto; 

means for filtering the illumination incident to said 
image sensing array so that at least a first group of 
said image sensing elements receives illumination 
within a first select range of wavelengths, a second 
select group of said image sensing elements re- 
ceives illumination within a second select range of 
wavelengths different from said first select range, 
and a third select group of said image sensing 
elements receives illumination within a third select 
range of wavelengths different from said first and 
second select ranges; 

means for interpolating the electronic information 
signals from said first group of image sensing 
elements for providing a first set of electronic in- 
formation signals corresponding to the intensity of 
illumination within said first range of wavelengths 
for said first, second, and third groups of image 
sensing elements, for interpolating the electronic 
information signals from said second group of im- 
age sensing elements for providing a second set of 
electronic information signals corresponding to the 
intensity of illumination within said second range of 
wavelengths for said first, second and third groups 
of image sensing elements, and for interpolating 
the electronic information signals from said third 
group of image sensing elements for providing a 
third set of electronic information signals corre- 
sponding to the intensity of illumination within said 
third range of wavelengths for said first, second, 
and third groups of image sensing elements, 
means for combining said first and second sets of 
electronic information signals to provide a first 
combined electronic information signal and median 
filtering said first combined electronic information 
signal to provide a first processed electronic in- 
formation signal and for combining said second 
and third sets of electronic information signals to 
provide a second combined electronic information 
signal and median filtering said second combined 
electronic information signal to provide a second 
processed electronic information signal; and 
means for combining said first and second pro- 
cessed electronic information signals for each im- 
age sensing element in said first group of image 
sensing elements with the electronic information 
signal originally sensed for that same image sens- 
ing element in said first group to provide first 
output electronic information signals corresponding, 
respectively, to the intensity of incident illumination 



within both said second and third range of 
wavelengths for each of said first group of image 
sensing elements, combining said first and second 
processed electronic information signals for each 

s image sensing element in said second group of 
image sensing elements with the electronic infor- 
mation signal originally sensed for that same image 
sensing element in said second group to provide 
second output electronic information signals cor- 

10 responding, respectively, to the intensity of incident 
illumination with both said first and third ranges of 
wavelengths, and for combining said first and sec- 
ond processed electronic information signals for 
each image sensing element in said third group of 

75 image sensing elements with the electronic infor- 
mation signal originally sensed for that same image 
sensing element in said third group to provide third 
output electronic information signals corresponding, 
respectively, to the intensity of incident illumination 

20 within both said first and second ranges of 
wavelengths for each of said third group of image 
sensing elements. 

2. The apparatus of claim 1 including a first 
gate circuit connected to receive electronic infor- 

25 mation signals originally sensed by said first group 
of image sensing elements as well as said output 
electronic information signals corresponding, re- 
spectively, to the intensity of incident illumination 
within said second and third range of wavelengths 

30 for each of said first group of image sensing ele- 
ments, a second gate circuit connected to receive 
electronic information signals originally sensed by 
said second group of image sensing elements as 
well as said output electronic information signals 

35 corresponding, respectively, to the intensity of in- 
cident illumination within said first and third ranges 
of wavelengths for each of said second group of 
image sensing elements, and a third" gate circuit 
connected to receive electronic information signals 

40 originally sensed by said third group of image 
sensing elements as well as said output electronic 
information signals corresponding, respectively, to 
the intensity of incident illumination within said first 
and second ranges of wavelengths for each of said 

45 third group of image sensing elements, said gate 
circuits each being selectively enableable to trans- 
mit therethrough each of the signals received 
thereby; and 

timing control means for providing a timing control 
so signal to selectively enable said gate circuits, one 
at a time, in correspondence with the receipt by 
the one of said gate circuits to be enabled of its 
originally sensed electronic information signal. 

3. The apparatus of claim 2 wherein said com- 
55 bining means comprises a plurality of adders and 

subtracters, a first subtracter of which is connected 
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to provide an output signal corresponding to the 
difference between said first and second processed 
electronic information signals. 

4. The apparatus of claim 3 wherein said ad- 
ders and subtracters of said combining means fur- 
ther comprise a first adder for providing one of said 
input signals to said first gate circuit corresponding 
to the sum of said first processed electronic in- 
formation signal and the electronic information sig- 
nal originally sensed by said first group of image 
sensing elements, a second adder for providing 
another of said input signals to said first gate 
circuit corresponding to the sum of said second 
processed electronic information signal and the 
electronic information signal originally sensed by 
said first group of image sensing elements, a third 
adder for providing one of said input signals to said 
second gate circuit corresponding to the sum of 
said output signal from said first subtracter and the 
electronic information signal originally sensed by 
said second group of image sensing elements, a 
second subtracter for providing another one of said 
input signals to said second gate circuit corre- 
sponding to the difference between said second 
processed electronic information signal and the 
electronic information signal originally sensed by 
said second group of image sensing elements, a 
third subtracter for providing one of said input 
signals to said third gate circuit corresponding to 
the difference between said first processed elec- 
tronic information signal and the electronic informa- 
tion signal originally sensed by said third group of 
image sensing elements, and a fourth subtracter for 
providing another one of the input signals to said 
third gate circuit corresponding to the difference 
between said output signal from said first subtrac- 
ter and the electronic information signal originally 
sensed by said third group of image sensing ele- 
ments. 

5. The apparatus of claim 1 wherein said 
means for median filtering comprises means for 
selecting the median combined electronic informa- 
tion signal for each image sensing element from a 
select plurality of combined electronic information 
signals each corresponding, respectively, to one of 
a plurality of select image sensing elements, said 
select image sensing elements being spaced apart 
with respect to each other by other non-select 
image sensing elements. 

6. The apparatus of claim 5 wherein said select 
image sensing elements are linearly arranged with 
respect to each other and comprise the first, fourth 
and seventh image sensing elements in a line of 
seven consecutively spaced image sensing ele- 
ments. 

7. The apparatus of claim 5 wherein said select 
image sensing elements are part of a two-dimen- 
sional array comprising five rows of seven image 



sensing elements in each row wherein said select 
image sensing elements comprise the fourth image 
sensing element of said first and fifth rows, the 
second and sixth image sensing elements of said 
5 second and fourth rows, respectively, and the first, 
fourth and seventh image sensing elements of said 
third row. 

8. The apparatus of claim 1 including means 
for low pass filtering said first and second com- 

io bined electronic information signals prior to median 
filtering and immediately subsequent to median 
filtering. 

9. In an electronic imaging apparatus of the 
type comprising an image sensing array including 

75 a predetermined number of discrete image sensing 
elements each responsive to incident illumination 
from a subject to provide an electronic information 
signal corresponding to the intensity of the illu- 
mination incident thereto; means for filtering the 
20 illumination incident to said image sensing array so 
that at least a first group of said image sensing 
elements receives illumination within a first select 
range of wavelengths and a second select group of 
said image sensing elements receives illumination 
25 within a second select range of wavelengths dif- 
ferent from said first select range; means for inter- 
polating the electronic information signals from said 
first group of image sensing elements for providing 
a first set of electronic information signals cor- 
30 responding to the intensity of illumination within 
said first range of wavelengths for said first and 
second groups and for interpolating the electronic 
information signals from said second group of im- 
age sensing elements for providing a second set of 
35 electronic information signals corresponding to the 
intensity of illumination within said second range of 
wavelengths for said first and second groups of 
image sensing elements, means for combining said 
first and second sets of electronic information sig- 
40 nals to provide a combined electronic information 
signal and thereafter median filtering said com- 
bined electronic information signal to provide a 
processed electronic information signal for each 
image sensing element, and means for combining 
45 the processed electronic information signal for 
each image sensing element in said first group of 
image sensing elements with the electronic infor- 
mation originally sensed for that same image sens- 
ing element in said first group to provide an output 
so electronic information signal corresponding to the 
intensity of incident illumination within said second 
range of wavelengths for each of said first group of 
image sensing elements, and for combining said 
processed electronic information signal for each 
55 image sensing element in said second group of 
image sensing elements with the electronic infor- 
mation signal originally sensed for that same sens- 
ing element in said second group to provide an- 
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other output electronic information signal corre- 
sponding to the intensity of incident illumination 
within said first range of wavelengths for each of 
said second group of image sensing elements, the 
improvement characterized by: 
said means for median filtering including means for 
selecting the median combined electronic informa- 
tion signal for each image sensing element from a 
select plurality of combined electronic information 
signals each corresponding, respectively, to one of 
a plurality of select image sensing elements, said 
select image sensing elements being spaced apart 
with respect to each other by other non-select 
image sensing elements. 

10. The apparatus of claim 9 wherein said 
select image sensing elements are linearly ar- 
ranged with respect to each other and comprise 
the first, fourth and seventh image sensing ele- 
ments in a line of seven consecutively spaced 
image sensing elements. 

11. The apparatus of claim 9 wherein said 
select image sensing elements are part of a two- 
dimensional array comprising five rows of seven 
image sensing elements in each row wherein said 
select image sensing elements comprise the fourth 
image sensing element of said first and fifth rows, 
the second and sixth image sensing elements of 
said second and fourth rows, respectively, and the 
first, fourth, and seventh image sensing elements 
of said third row. 

12. A method of sensing a subject and provid- 
ing an output from which a visible image of the 
subject may be constructed comprising the steps 
of: 

sensing light from the subject with an image sens- 
ing array comprising a predetermined number of 
discrete image sensing elements each of which 
responds to incident illumination from the subject 
to provide an output corresponding to the intensity 
of the illumination incident thereto; 
filtering the illumination incident to the image sens- 
ing array so that at least a first group of image 
sensing elements receives illumination within a first 
select range of wavelengths, a second group of 
image sensing elements receives illumination with- 
in a second select range of wavelengths and a third 
group of image sensing elements receives illumina- 
tion within a third select range of wavelengths; 
interpolating the outputs from the first group of 
image sensing elements to provide a first set of 
outputs corresponding to the intensity of illumina- 
tion within the first range of wavelengths for said 
first, second, and third groups of image sensing 
elements; 

interpolating the outputs from the second group of 
image sensing elements to provide a second set of 
outputs corresponding to the intensity of illumina- 
tion within the second range of wavelengths for 



said first, second, and third groups of image sens- 
ing elements; 

interpolating the outputs from the third group of 
image sensing elements to provide a third set of 
5 outputs corresponding to the intensity of illumina- 
tion within the third range of wavelengths for said 
first, second, and third groups of image sensing 
elements; 

combining said first and second sets of outputs to 

70 provide a first combined output; 

median filtering said first combined output to pro- 
vide, a first processed output corresponding to the 
median value of said first combined output; 
combining said second and third sets of outputs to 

15 provide a second combined output; 

median filtering said second combined output to 
provide a second processed output corresponding 
to the median value of said second combined 
output; 

20 combining said first and second processed outputs 
for each image sensing element in said first group 
of image sensing elements with the output original- 
ly sensed for that same image sensing element in 
said first group to provide first output electronic 

25 information signals corresponding, respectively, to 
the intensity of illumination within both said second 
and third ranges of wavelengths for each of said 
first groups of image sensing elements; 
combining said first and second processed outputs 

30 for each image sensing element in said second 
group of image sensing elements with the output 
originally sensed for that same image sensing ele- 
ment in said second group to provide second out- 
put electronic information signals corresponding, 

35 respectively, to the intensity of illumination within 
both said first and third ranges of wavelengths; and 
combining said first and second processed outputs 
for each image sensing element in said third group 
of image sensing elements with the output original- 

40 ly sensed for that same image sensing element in 
said third group to provide third output electronic 
information signals corresponding, respectively, to 
the intensity of illumination within both said first 
and second ranges of wavelengths. 

45 13. The method of claim 12 further comprising 

the steps of: 

gating through the output originally sensed by said 
first group of image sensing elements and said first 
output electronic information signals in correspon- 

50 dence with receipt of the output originally sensed 
by said first group of image sensing elements; 
gating through the output originally sensed by said 
second group of image sensing elements and said 
second output electronic information signal in cor- 

55 respondence with receipt of the output sensed by 
said second group of image sensing elements; and 
gating through the output originally sensed by said 
third group of image sensing elements and said 
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third output electronic information signal in cor- 
respondence with receipt of the output originally 
sensed by said third group of image sensing ele- 
ments. 

14. The method of claim 13 wherein said step 
of combining said first and second outputs with the 
output originally sensed for that same image sens- 
ing element in said first group comprises the steps 
of: adding said first processed output and the out- 
put originally sensed by said first group of image 
sensing elements to provide said first output elec- 
tronic information signal corresponding to the inten- 
sity of illumination within said second range of 
wavelengths; adding said second processed output 
and the output originally sensed for said first group 
of image sensing elements to provide said first 
output electronic information signal corresponding 
to the intensity of illumination within said third 
range of wavelengths; providing a difference output 
corresponding to the difference between said first 
and second processed outputs; adding said dif- 
ference output and the output originally sensed by 
said second group of image sensing elements to 
provide said second output electronic information 
signal corresponding to the intensity of illumination 
within said first range of wavelengths; taking the 
difference between said second processed output 
and the output originally sensed by said second 
group of image sensing elements to provide said 
second output electronic information signal cor- 
responding to the intensity of illumination within 
said third range of wavelengths, taking the dif- 
ference between said first processed output and 
the output originally sensed by said third group of 
image sensing elements to provide said third out- 
put electronic information signal corresponding to 
the intensity of illumination within said first range of 
wavelengths; and taking the difference between 
said difference output and the output originally 
sensed by said third group of image sensing ele- 
ments to provide said third output electronic in- 
formation signal corresponding to the intensity of 
illumination within said second range of 
wavelengths. 

15. The method of claim 12 wherein each of 
said steps for median filtering comprises determin- 
ing the median value for each image sensing ele- 
ment from a select plurality of combined outputs 
each corresponding, respectively, to one of a plu- 
rality of select image sensing elements and select- 
ing said select image sensing elements to be 
spaced apart with respect to each other by non- 
select image sensing elements. 

16. The method of claim 15 wherein said step 
of selecting said select image sensing element 
comprises selecting the first, fourth and seventh 
image sensing elements from a line of seven con- 
secutively spaced image sensing elements. 



17. The method of claim 15 wherein said step 
of selecting said select image sensing element 
comprises first designating a two-dimensional array 
having five rows with seven image sensing ele- 

5 ments in each row and thereafter selecting the 
fourth image sensing element of said first and fifth 
rows, the second and sixth image sensing ele- 
ments of said second and fourth rows, respectively, 
and the first, fourth and seventh image sensing 

io elements of said third row. 

18. The method of claim 1 including the step of 
low pass filtering said first and second combined 
outputs prior to median filtering and immediately 
subsequent to median filtering. 

76 19. A method of sensing a subject and provid- 

ing an output from which a visible image of the 
subject may be constructed comprising the steps 
of: 

sensing light from the subject with an image sens- 

20 ing array comprising a predetermined number of 
discrete image sensing elements each of which 
responds to incident illumination from the subject 
to provide an output corresponding to the intensity 
of the illumination incident thereto; 

25 filtering the illumination incident to the image sens- 
ing array so that at least a first group of said image 
sensing elements receives illumination within a first 
select range of wavelengths and a second select 
group of image sensing elements receives illumina- 

30 tion within a second select range of wavelengths 
different from said first select range; 
interpolating the output from said first group of 
image sensing elements for providing a first set of 
outputs corresponding to the intensity of illumina- 

35 tion within the first range of wavelengths for said 
first and second groups of image sensing ele- 
ments; 

interpolating the output from said second group of 
image sensing elements for providing a second set 
40 of outputs corresponding to the intensity of illu- 
mination within the second range of wavelengths 
for said first and second groups of image sensing 
elements; 

combining said first and second sets of outputs to 

45 provide a first combined output: 

median filtering said first combined output by de- 
termining the median value for each image sensing 
element from a select plurality of combined outputs 
each corresponding, respectively, to one of a plu- 

50 rality of select image sensing elements and select- 
ing said select image sensing elements to be 
spaced apart with respect to each other by non- 
select image sensing elements; 
combining the median filtered output for each im- 

55 age sensing element in said first group of image 
sensing elements with the output originally sensed 
for that same image sensing element in said first 
group to provide a first output electronic informa- 

10 
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tion signal corresponding, respectively, to the in- 
tensity of illumination within said second range of 
wavelengths for each of said first group of image 
sensing elements; and 

combining the median filtered output for each im- 5 
age sensing element in said second group of im- 
age sensing elements with the output originally 
sensed for that same image sensing element in 
said second group to provide a second output 
electronic information signal corresponding, re- 1Q 
spectively, to the intensity of illumination within 
said first range of wavelengths for each of said 
second groups of image sensing elements. 

20. The method of claim 19 wherein said step 

of selecting said select image sensing elements is 
comprises selecting the first, fourth and seventh 
image sensing elements from a line of seven con- 
secutively spaced image sensing elements. 

21. The method of claim 19 wherein said step 

of selecting said select image sensing elements 20 
comprises first designating a two-dimensional array 
having five rows with seven image sensing ele- 
ments in each row and thereafter selecting the 
fourth image sensing element of said first and fifth 
rows, the second and sixth image sensing ele- 25 
ments of said second and fourth rows, respectively, 
and the first, fourth and seventh image sensing 
elements of said third row. 
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